On Stationary Markov Processes
نویسندگان
چکیده
منابع مشابه
Stationary Markov Processes
We consider some classes of stationary, counting{measure{valued Markov processes and their companions under time{reversal. Examples arise in the L evy{It^ o decomposition of stable Ornstein{Uhlenbeck processes, the large{time asymptotics of the standard additive coalescent, and extreme value theory. These processes share the common feature that points in the support of the evolving counting{ me...
متن کاملMarkov processes with product-form stationary distribution
This research has been inspired by several papers on processes with inert drift [5, 6, 4, 3, 1]. The model involves a “particle” X and an “inert drift” L, neither of which is a Markov process by itself, but the vector process (X, L) is Markov. It turns out that for a number of diffusions with inert drift, the stationary measure has the product form; see [1]. The purpose of this note is to chara...
متن کاملBounding Stationary Expectations of Markov Processes
This paper develops a simple and systematic approach for obtaining bounds on stationary expectations of Markov processes. Given a function f which one is interested in evaluating, the main idea is to find a function g that satisfies a certain “mean drift” inequality with respect to f , which in turn leads to bounds on the stationary expectation of the latter. The approach developed in the paper...
متن کاملON THE INFINITE ORDER MARKOV PROCESSES
The notion of infinite order Markov process is introduced and the Markov property of the flow of information is established.
متن کاملOn the Use of Non-Stationary Policies for Stationary Infinite-Horizon Markov Decision Processes
We consider infinite-horizon stationary γ-discounted Markov Decision Processes, for which it is known that there exists a stationary optimal policy. Using Value and Policy Iteration with some error ǫ at each iteration, it is well-known that one can compute stationary policies that are 2γ (1−γ)2 ǫ-optimal. After arguing that this guarantee is tight, we develop variations of Value and Policy Iter...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: The Annals of Mathematical Statistics
سال: 1967
ISSN: 0003-4851
DOI: 10.1214/aoms/1177698975